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It is the responsibility of the applicant or the attorney or agent to make the substance of an interview of record in the application file, unless 
the examiner indicates he or she will do so. It is the examiner's responsibility to see that such a record is made and to correct material inaccuracies 
which bear directly on the question of patentability. 
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- Name of participant(s) (applicant, attorney or agent, examiner, other PTO personnel, etc.) 
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- An identification of the specific prior art discussed 
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- The signature of the examiner who conducted the interview (if Form is not an attachment to a signed Office action) 

It is desirable that the examiner orally remind the applicant of his or her obligation to record the substance of the interview of each case. It 
should be noted, however, that the Interview Summary Form will not normally be considered a complete and proper recordation of the interview 
unless it includes, or is supplemented by the applicant or the examiner to include, all of the applicable items required below concerning the 
substance of the interview. 

A complete and proper recordation of the substance of any interview should include at least the following applicable items: 

1) A brief description of the nature of any exhibit shown or any demonstration conducted, 

2) an identification of the claims discussed, 

3) an identification of the specific prior art discussed, 

4) an identification of the principal proposed amendments of a substantive nature discussed, unless these are already described on the 
Interview Summary Form completed by the Examiner, 

5) a brief identification of the general thrust of the principal arguments presented to the examiner, 

(The identification of arguments need not be lengthy or elaborate. A verbatim or highly detailed description of the arguments is not 
required. The identification of the arguments is sufficient if the general nature or thrust of the principal arguments made to the 
examiner can be understood in the context of the application file. Of course, the applicant may desire to emphasize and fully 
describe those arguments which he or she feels were or might be persuasive to the examiner.) 

6) a general indication of any other pertinent matters discussed, and 

7) if appropriate, the general results or outcome of the interview unless already described in the Interview Summary Form completed by 
the examiner. 

Examiners are expected to carefully review the applicant's record of the substance of an interview. If the record is not complete and 
accurate, the examiner will give the applicant an extendable one month time period to correct the record. 

Examiner to Check for Accuracy 

If the claims are allowable for other reasons of record, the examiner should send a letter setting forth the examiner's version of the 
statement attributed to him or her. If the record is complete and accurate, the examiner should place the indication, "Interview Record OK" on the 
paper recording the substance of the interview along with the date and the examiner's initials. 
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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 

Serial No 10/815,140 

Filing Date Mar 31, 2004 

First Named Inventor John Michael Snyder 

Assignee Microsoft Corporation 

Group Art Unit 2628 

Examiner Jason M. Repko 

Attorney's Docket No MS1-3541US 

Title ........Systems and Methods for All-Frequency Relighting Using Spherical 

Harmonics and Point Light Distributions 

Informal Communication in Preparation for 
Scheduling an Interview 



To: Examiner Repko 

Fax: (571) 273-8624 



From: Jason F. Lindh 

Lee & Hayes, PLLC 
421 W. Riverside Avenue, Suite 500 
Spokane, WA 99201 
jason@leehayes.com 
• (Tel. 509-324-9256; Fax 509-323-8979) 



Dear Examiner Repko: 

[0001] This communication provides an agenda for a phone interview 
of this matter. My assistant will be contacting you to schedule an interview. 
If you would prefer to schedule the interview, then please contact my 
assistant or me directly. Our contact info is on the signature page of this 
document. Thank you in advance to talking with me about this matter. 
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Interview Agenda: 

■ Discussion about the Abstract 

■ Discussion about the Specification 

■ Discussion of the Drawings 

■ Discussion of the § 101 issues 

■ Discussion of the § 112, Second Paragraph, issues 

■ Discussion the new claims; 

■ Discussion of proposed amendments 

The Abstract 

[0002] The abstract has been objected to because the original abstract 
was not sufficiently clear and concise. The applicant proposes the following 
amended abstract: 



The-present-inven t ion is dire ct e d t o Computer 
graphics systems and methods for all-frequency 
relighting are described. In one described 
embodiment, all-frequency relighting is 
achieved by representing low frequencies of 
lighting with spherical harmonics and 
[approximate] approximating the residual high- 
frequency energy with point lights. In another 
One such embodiment [ r e nd e rs ] low-frequencies 
are rendered with precomputed radiance 
transfer (PRT) techniques ( w hic h requ i res on l y 
a^oder-ate-amouf^f-^eeemputa tion and 
storage-);- while the higher-frequencies are 
rendered with on-the-fly techniques such a s 
shadew^aps^nd-shadow^elumes. In a ddition, 
various Further embodiments are directed [te] 
towards a systems and methods for 
decomposing the lighting into harmonics and sets 
of point lights. Var i ous altern a tive e mbod i ments 
are-d ireeted-to-systems-and-metho ds for 
eharaeterizing-th c types o f^nviren ments for 
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[0003] With this amendment to the abstract, the applicant believes 
that the reasoning for the objection of the abstract is no longer applicable. 

The Specification 

[0004] The specification has been objected to because of the inclusion 
of the Attorney Docket number in the [0002] paragraph of the disclosure. 
The Applicant proposes amending the specification by replacing paragraphs 
[0001] and [0002] with the following. 



[0001] This application claims benefit of U.S. 
Provisional Patent Application No. 60/510,191 
[ (Cl i ent Docket No. 306 8 75,01) ], filed on 

October 10, 2003, entitled "ALL FREQUENCY 
RELIGHTING USING SPHERICAL HARMONICS 
AND POINT LIGHT DISTRIBUTION," the contents 
of which are hereby incorporated herein by 
reference in their entirety. 

[0002] This application is related by subject 
matter to the inventions disclosed in the following 
commonly assigned applications, the entire 
contents of which are hereby incorporated herein 
by reference: U.S. Patent Application No. 
10/815,141 [ (not yet assign e d) (Atty. 
Poefcefc-N o. MSFT - 2001/30687 4 .02) ], filed 
on even date herewith, entitled "SYSTEMS AND 
METHODS FOR ROBUST SAMPLING FOR REAL- 
TIME RELIGHTING OF OBJECTS IN NATURAL 
LIGHTING ENVIRONMENTS." 




techniqu e i n torrr 
qual ity-£FeaJism)r 
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[0005] With these proposed amendment to the specification, the 
applicant believes that the objection to the specification would no longer be 
applicable. 

The Drawings 

[0006] The drawings, specifically figure 1, has been objected to as 
failing to comply with 37 C.F.R. § 1.84(p)(5) because the figure includes a 
reference character 36' which is not mentioned in the description. The 
applicant proposes amending the drawings by replacing the reference 
character 36' with the reference character 36. The applicant believes that 
this proposed amendment to the drawing would be sufficient to overcome 
the objection. 

The 35 U.S.C. § 101 Rejection 

[0007] Claims 1-9 have been rejected as being directed to a process 
consisting solely of operation manipulating a set of mathematical entities. 
Claims 1 has been amended to include the element that the method is a 
method for ''displaying all-frequency relighting of computer-generated 
graphic objects' 7 and that the method includes "displaying the computer 
graphic objects". The Applicant believes that this amendment addresses the 
concerns of the Examiner, and that further the amended claim now claims 
subject matter capable of producing a useful, concrete and tangible result. 
Please see the attached appendix with the complete list of amendments to 
each of the claims. 
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[0008] Claims 10-18 and 28-30 have been rejected as being directed to 
encompass a computer program not technologically embodied to enable the 
functionality to be realized. The applicant proposes amending paragraph 
[0018] of the specification in the following manner: 

[0003] While it is envisioned that numerous 
embodiments of the present invention are 
particularly well-suited for computerized systems, 
nothing in this document is intended to limit the 
invention to such embodiments. On the contrary, 
as used herein the term "computer system" is 
intended to encompass any and all devices 
capable of storing and processing information 
and/or capable of using the stored information to 
control the behavior or execution of the device 
itself, regardless of whether such devices are 
electronic or mechanical [ , log i ca l , or v i rtua l i n 
nature]. 

[0009] The applicant believes that this proposed amendment to the 
specification would be sufficient to overcome the rejection of claims 10-18 
and 28-30. Additionally, claims 10 and 28 have been amended to include 
elements that clarify that the claims claim patent eligible subject matter that 
is capable of producing a useful, concrete and tangible result. 

[0010] Claims 19-27 have been rejected for reciting a computer- 
readable medium comprising computer-readable instructions that includes 
non-functional descriptive material encoded on a computer readable 
medium. Claim 19 has been amended to claim, per the suggestion of the 
Examiner, that the computer-readable instructions are stored on the 
computer-readable medium. Claim 19 is now in proper form for allowance. 
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The 35 U.S.C § 112, second paragraph 

[0011] Claims 7-9 and 25-27 have been rejected as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which 
the applicant regards as the invention. Claims 7-9 and 25-27 have been 
canceled and incorporated into their respective independent claims, thus the 
§ 112, second paragraph, at moot. 

New Claims 

[0012] Claims 31-45 are proposed new claims. Claims 31 is similar in 
scope to independent claim 1 that incorporates the elements of canceled 
claim 8 instead of claim 7. Claim 36 is similar in scope to independent claim 
10 that incorporates the elements of canceled claim 17. Claim 41 is similar 
in scope to that of independent claim 19 that incorporates the element of 
canceled claim 26. As the Examiner has indicated that these dependent 
claims would be allowable if written in independent form, the applicant 
respectfully suggests that tfiese claims are now in proper form for allowance. 

Proposed Amendments 

[0013] Each of the proposed amended independent claims include 
subject matter that the Examiner has indicated to be allowable if the § 101 
issues were addressed. 
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Conclusion 



[0014] Once again, thank you for taking the time to talk with me. I 
look forward to talking to you. 



Respectfully Submitted, 



Jason F. Lindh 
Reg. No. 59090 
(509) 324-9256 x215 
jason@leehayes .com 
www.leehayes.com 



My Assistant: Carly Bokarica 
(509) 324-9256 x264 
carly@leehayes.com 
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Appendix of Claims with Proposed Amendments 
What is Claimed: 

1. (Currently Amended) A method for displaying all- 
frequency rejighting of computer-generated graphic objects, 
comprising: 

using approximations based on an radiance accumulation derived 
from a combination of at least two real-time techniques; [ and wherein 
a] 

representing a set of low frequencies of lighting [are 
represented ] with a tabulated rendering method; [and] 

approximatin g high-frequency energy [i s approx i mated ] with 
an on -the -fly method; 

segmenting a lighting environment into a plurality of 
regions; 

prioritizing each of the plurality of regions for subsequent 
extraction and approximation with a plurality of small analytic 
light sources, wherein said element of prioritizing each of the 
plurality of reg ions f or subsequent extraction and approximation 
with point light sources comprises is based on high-frequency 
energy reduc t ; on; 

allocating one or more point samples to each of said 
plurality of regions; and 

display ing the computer-generated graphic objects. 
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2. (Original) The method of claim 1 wherein a set of low frequencies 
of lighting are represented with a precomputed radiance transfer (PRT) 
technique and high-frequency energy is approximated with a plurality of 
point lights. 

3. (Original) The method of claim 0 wherein a set of low frequencies 
of lighting is rendered using a spherical harmonics technique. 

4- (Original) Tie method of claim 0 wherein a set of high 
frequencies of lighting is rendered using a shadow map technique. 

5. (Original) The method of claim 0 wherein a set of high 
frequencies of lighting is rendered using a shadow volumes technique. 

6. (Canceled) 

7. (Canceled) 
8- (Canceled) 

9. (Canceled) 

10. (Currently Amended) A system for displaying all-frequency 
relighting of computer-generated graphic objects, comprising: 

a subsystem for determining approximations based on an radiance 
accumulation derived from a combination of at least two real-time 
techniques [ comp r4smg4]; 

The Business of IP 
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a subsystem for representing a set of low frequencies of lighting 
with a tabulated rendering method; [and] 

a subsystem for approximating high-frequency energy using an on- 
the-fly method; 

a subsystem for segmenting a lighting environment into a 
plurality of regions; 

a subsystem for prioritizing each of the plurality of regions 
for subseguent extraction and approximation .with a plurality of 
small analytic light sources, wherein said subsystem for 
prioritizing each of the plurality of regions for subsequent 
extraction and approximation with point light sources utilizes a 
high-freguency energy reduction technique; 

a subsystem for allocating one or more point samples to 
each of said p^irpli^y of regions; and 

a subsy stem for dis playing the computer-generated graphic 
objects. 

11. (Original) The system of claim 10 further comprising a subsystem 
for representing a set of low frequencies of lighting with precomputed 
radiance transfer (PRT) and a subsystem for approximating high-frequency 
energy with a p'urality of point lights. 

12. (Original) The system of claim 11 further comprising a subsystem 
using a spherical harmonics technique to render a set of low frequencies of 
lighting. 
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13. (Original) The system of claim 11 further comprising a subsystem 
using a shadow map technique to render a set of high frequencies of 
lighting. 

14. (Original) The system of claim 11 further comprising a subsystem 
using a shadow volumes technique to render a set of high frequencies of 
lighting. 

15. (Canceled) 

16. (Canceled) 

17. (Canceled) 

18. (Canceled) 

19. (Currently Amended) A computer-readable medium storing 
[eempriswg] computer-readable instructions for displaying all- 
frequency relighting of computer-generated graphic objects, [using 
approxi matk*ns~based-o n an r a d i ance accumu l at i on d e r i ved from 
a combinatio nH>f-^a t-4eas^t\vo rea l- t i me technique s , ] said computer 
readable instructions comprising instructions for: 

using approximations based on an radiance accumulation 
derived from ? cor^^n^\nn of at least two real-time techniques; 

representing a set of low frequencies of lighting with a tabulated 
rendering method; [amf] 

ap p roxi m^tinn h?nh-frequency energy with an on-the-flv 
method; 
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segmenting a lighting environment into a plurality of 
regions; 

prioritizing each of the plurality of regions for subsequent 
extraction and approximation with a plurality of small analytic 
light sources, whereby said element of prioritizing each of the 
plurality of regions for subsequent extraction and approximation 
with point light sources comprises is based on high-frequency 
energy rerjp « -t ; nn; 

allocating one or more point samples to each of said 
plurality of rp m^ns; and 

displayin g thg rompn >:er-generated graphic objects, 

20. (Original) The computer-readable instructions of claim 19 further 

comprising instructions whereby a set of low frequencies of lighting are 
represented vi'-h a precomputed radiance transfer (PRT) technique and 
high-frequency energy is approximated with a plurality of point lights. 

21. (Orgin? 1 ) The computer-readable instructions of claim 20 further 
comprising instructions whereby a set of low frequencies of lighting is 
rendered using a spherical harmonics technique. 

22. (Origin*!) Tine computer-readable instructions of claim 20 further 

comprising instructions whereby a set of high frequencies of lighting is 
rendered using a shadow map technique. 

23. (Originr!) Tie computer-readable instructions of claim 20 further 
comprising instructions whereby a set of high frequencies of lighting is 
rendered using a shadow volumes technique. 
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24. (Canceled) 

25. (Canceled) 

26. (Canceled) 

27. (Canceled) 

28. (Currently Amended) A hardware control device for displaying 
all-frequency relighting of computer -generated graphic objects using 
approximations based on an radiance accumulation derived from a 
combination of at least two real-time techniques, said device comprising: 

means by which a set of low frequencies of lighting are represented 
with a precomp iled radiance transfer (PRT) technique and high-frequency 
energy is approximated with a plurality of point lights; 

means by which a set of high frequencies of lighting is rendered 
using one of a ret of techniques, said set of techniques comprising: a 
shadow map technique, and a shadow volumes technique 

means f or segmenting a lighting environment into a 
plurality of r^^ta regions ; 

pi ip *r\s ^ r prioritizing each of the plurality of regions for 
subsequen r p Taction and approximation with a plurality of small 
analytic [icjh*- -pu ree s, w herein said means for prioritizing each of 
the ph 'ralif y of regions for subsequent extraction and 
approximat?*™ LV l tb point light sources comprises means 
emplo ying a fr^r.hmgu q based on one of the following sets of 
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techniques: a high-frequency energy reduction technique or an 
iterative algorithm for a background estimation technique; 

means for allocating one or more point samples to each of 
said plurality of regions; and 

means for displaying the computer-generated graphic 
objects. 

29. (^pnceler) 

30. (Canceled) 

31. (New) A method for displaying all-frequency relighting of 
computer-generated graphic objects, comprising: 

using approximations based on an radiance accumulation derived . 
from a combination of at least two real-time techniques; 

representing a set of low frequencies of lighting with a tabulated 
rendering method; 

approximating high-frequency energy with an on-the-fly method; 

segmenting a lighting environment into a plurality of regions; 

prioritizing each of the plurality of regions for subsequent extraction 
and approximation with a plurality of small analytic light sources, wherein 
said element of prioritizing each of the plurality of regions for subsequent 
extraction and approximation with point light sources comprises is based 
on an iterative algorithm for a background estimation; 

allocating one or more point samples to each of said plurality of 
regions; and 

displaying the computer-generated graphic objects. 
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32. (New) The method of claim 31 wherein a set of low 
frequencies of lighting are represented with a precomputed radiance 
transfer (PRT) technique and high-frequency energy is approximated with 
a plurality of point lights. 

33. (New) The method of claim 32 wherein a set of low 
frequencies of lighting is rendered using a spherical harmonics technique. 

34. (New) Tine method of claim 32 wherein a set of high 
frequencies of lighting is rendered using a shadow map technique. 

35. (New) The method of claim 32 wherein a set of high 
frequencies of lighting is rendered using a shadow volumes technique. 
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36. (New) A system for displaying all-frequency relighting of 
computer-generated graphic objects, comprising: 

a subsystem for determining approximations based on an radiance 
accumulation derived from a combination of at least two real-time 
techniques; 

a subsystem for representing a set of low frequencies of lighting 
with a tabulated rendering method; 

a subsystem for approximating high-frequency energy using an on- 
the-fly method; 

a subsystem for segmenting a lighting environment into a plurality 
of regions; 

a subsystem for prioritizing each of the plurality of regions for 
subsequent extraction and approximation with a plurality of small analytic 
light sources, wherein said subsystem for prioritizing each of the plurality 
of regions for subsequent extraction and approximation with point light 
sources utilizes an iterative algorithm for a background estimation 
technique; 

a subsystem for allocating one or more point samples to each of 

said plurality of regions; and 

a subsystem for displaying the computer-generated graphic objects. 
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37. (New) The system of claim 36 further comprising a subsystem 
for representing a set of low frequencies of lighting with precomputed 
radiance transfer (PRT) and a subsystem for approximating high-frequency 
energy with a plurality of point lights. 

38. (New) The system of claim 37 further comprising a subsystem 
using a spherical harmonics technique to render a set of low frequencies of 
lighting. 

39- (New) The system of claim 37 further comprising a subsystem 
using a shadow map technique to render a set of high frequencies of 
lighting. 

40. (New) Tine system of claim 37 further comprising a subsystem 
using a shadow volumes technique to render a set of high frequencies of 
lighting. 

41. (New) A computer-readable medium storing computer- 
readable instructions for displaying all-frequency relighting of computer- 
generated graphic objects, said computer readable instructions comprising 
instructions for: 

using approximations based on an radiance accumulation derived 
from a combination of at least two real-time techniques; 

representing a set of low frequencies of lighting with a tabulated 
rendering method; 

approximating high-frequency energy with an on-the-fly method; 

segmenting a lighting environment into a plurality of regions; 
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prioritizing each of the plurality of regions for subsequent extraction 
and approximation with a plurality of small analytic light sources, whereby 
said element of prioritizing each of the plurality of regions for subsequent 
extraction and approximation with point light sources comprises is based 
on an iterative algorithm for a background estimation; 

allocating one or more point samples to each of said plurality of 
regions; and 

displaying the computer-generated graphic objects. 

42. (N ew) The computer-readable instructions of claim 41 further 
comprising instructions whereby a set of low frequencies of lighting are 
represented with a precomputed radiance transfer (PRT) technique and 
high-frequency energy is approximated with a plurality of point lights. 

43. (New) The computer-readable instructions of claim 42 further 
comprising instructions whereby a set of low frequencies of lighting is 
render^ 1 u^ing a spherical harmonics technique. 

44. (N<3w) The computer-readable instructions of claim 42 further 
comprising instructions whereby a set of high frequencies of lighting is 
rendered using a shadow map technique. 

45. (rew) The computer-readable instructions of claim 42 further 
comppqnq instructions whereby a set of high frequencies of lighting is 
rendered using a shadow volumes technique. 
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